4 research outputs found
Precursor-of-Anomaly Detection for Irregular Time Series
Anomaly detection is an important field that aims to identify unexpected
patterns or data points, and it is closely related to many real-world problems,
particularly to applications in finance, manufacturing, cyber security, and so
on. While anomaly detection has been studied extensively in various fields,
detecting future anomalies before they occur remains an unexplored territory.
In this paper, we present a novel type of anomaly detection, called
\emph{\textbf{P}recursor-of-\textbf{A}nomaly} (PoA) detection. Unlike
conventional anomaly detection, which focuses on determining whether a given
time series observation is an anomaly or not, PoA detection aims to detect
future anomalies before they happen. To solve both problems at the same time,
we present a neural controlled differential equation-based neural network and
its multi-task learning algorithm. We conduct experiments using 17 baselines
and 3 datasets, including regular and irregular time series, and demonstrate
that our presented method outperforms the baselines in almost all cases. Our
ablation studies also indicate that the multitasking training method
significantly enhances the overall performance for both anomaly and PoA
detection.Comment: KDD 2023 accepted pape
EXIT: Extrapolation and Interpolation-based Neural Controlled Differential Equations for Time-series Classification and Forecasting
Deep learning inspired by differential equations is a recent research trend
and has marked the state of the art performance for many machine learning
tasks. Among them, time-series modeling with neural controlled differential
equations (NCDEs) is considered as a breakthrough. In many cases, NCDE-based
models not only provide better accuracy than recurrent neural networks (RNNs)
but also make it possible to process irregular time-series. In this work, we
enhance NCDEs by redesigning their core part, i.e., generating a continuous
path from a discrete time-series input. NCDEs typically use interpolation
algorithms to convert discrete time-series samples to continuous paths.
However, we propose to i) generate another latent continuous path using an
encoder-decoder architecture, which corresponds to the interpolation process of
NCDEs, i.e., our neural network-based interpolation vs. the existing explicit
interpolation, and ii) exploit the generative characteristic of the decoder,
i.e., extrapolation beyond the time domain of original data if needed.
Therefore, our NCDE design can use both the interpolated and the extrapolated
information for downstream machine learning tasks. In our experiments with 5
real-world datasets and 12 baselines, our extrapolation and interpolation-based
NCDEs outperform existing baselines by non-trivial margins.Comment: main 8 page
Learnable Path in Neural Controlled Differential Equations
Neural controlled differential equations (NCDEs), which are continuous analogues to recurrent neural networks (RNNs), are a specialized model in (irregular) time-series processing. In comparison with similar models, e.g., neural ordinary differential equations (NODEs), the key distinctive characteristics of NCDEs are i) the adoption of the continuous path created by an interpolation algorithm from each raw discrete time-series sample and ii) the adoption of the Riemann--Stieltjes integral. It is the continuous path which makes NCDEs be analogues to continuous RNNs. However, NCDEs use existing interpolation algorithms to create the path, which is unclear whether they can create an optimal path. To this end, we present a method to generate another latent path (rather than relying on existing interpolation algorithms), which is identical to learning an appropriate interpolation method. We design an encoder-decoder module based on NCDEs and NODEs, and a special training method for it. Our method shows the best performance in both time-series classification and forecasting